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corresponding  ti«r>  required  in  Making  tin-  dec  i i  on  '  we  re¬ 
recorded  lutuacunally. 


(■Mediate  feedback  on  each  decision  was  provided  m  order 


to  sustain  the  readers'  interest  and  to  help  the*  achieve  and 


Maintain  stable  performance.  The  computer  program  responded 


lows: 


response 


lesion  present 


to  each  entry  as  fot 
decision  rating 


1 

2 

3 

4 

s 


"you  are  correct" 
“you  are  correct " 
“there  is  a  lesion" 
"(here  is  a  lesion” 
"there  is  a  lesion" 


lesion  absent 

"there  is  no  lesion" 
"there  is  no  lesion" 
"there  is  no  lesion" 
"you  arc  correct" 
"you  are  correct" 


Phi*  AWM.VSIS 

The  diagnostic  performance  of  the  readers  using  both 
of  the  two  radiographic  nodal  it  tes  was  evaluated  by  ROC  (receiver 
opera! lag  characteristic)  analysis.  HOC  analysis  provides  an 
lades  of  diagnostic  accuracy  that  is  independent  of  extra- tnage 
decision  factors  and  prior  prohibit Ity  of  lesion  occurrence.7 
Speclflcal ly,  for  a  diagnostic  system  with  given  discriminatory 
capacity,  the  HOC  curve  shows  the  trading  relationship  between 
the  proportions  of  true-posit ive  (TPi  and  false-positive  (FP) 
responses,  as  the  decision  criteria  to  call  the  findings  positive 
or  negative  is  varied  systematically.  In  our  particular 
application,  this  graph  can  he  assessed  from  the  loci  or  points 
describing  the  relative  TP  and  FP  decisions  that  would  be  made 
by  considering  each  boundary  between  the  five  ctanincr  choices 
a  different  decision  criterion.  The  above  procedure  provided 


four  |>os!iible  points  that  arc  located  on  a  conceptually  smooth 
curve  characterizing  the  discrimination  capacity  of  a  particular 

O 

Modality.  A  commercial ly  available  computer  program  (RSCORE) 
used  to  fit  an  ROC  curve  through  the  four  empirically 
obtained  data  points.  The  theoretical  curve  is  based  on  the 
assumption  that  the  distributions  of  the  psychologically  perceived 

signal  strengths  in  the  presence  or  absence  of  a  lesion  are 

7  g 

normal.  *  Consistent  with  this  assumption,  the  data  points 
can  he  plotted  on  double  probability  (binormal)  coordinates**** 
and  fitted  by  a  straight  line.  The  computer  program  provided 
Measures  of  goodness-of-fit  of  that  line  and  a  maximum- likelihood 
eslinnte  of  an  ROC  index  of  diagnostic  accuracy,  A  ,  as  well 

Z 

as  it*  corresponding  sampling  variance.  The  index  A  reflects 

(he  location  of  the  entire  ROC  curve  rather  than  any  particular 

operating  point  thereon.  A  is  defined  by  the  area  beneath 

z 

the  fitted  ROC  curve,  and  ranges  from  a  minimum  of  0.5  for  chance 
performance  to  a  Maximum  of  1.0  for  perfect  discrimination 

capability. 

In  order  to  suanarize  the  performances  achieved  with  each 

Modality,  the  accuracy  indices,  A  ,  estimated  from  each  of 

z 

the  eteniners*  responses  were  either  pooled  or  averaged, 
the  associated  standard  errors  were  obtained  from  the  sampling 
eofiance*  of  the  maximum- likelihood  estimates  given  by  the 
ommputer  program.  The  statistical  significance  of  the  observed 


Chart  T4231,  Codex  Book  Company,  Norwood,  MA  02062 


difference  in  A^  between  the  two  modalities,  and  between  groups 

of  lesion  sites  with  comparable  anatomic  obscuration  was 

tested  by  a  paired  comparison.  This  was  possible  because  each 

reader  participated  in  the  evaluation  of  both  modalities. 

A  non-parametric  test  (sign  test)  was  preferred  in  view  of 

the  limited  range  of  A  and  the  small  number  of  readers  which 

z 

renders  the  normality  assumption  questionable. 

The  time  intervals  required  in  making  decisions  were 
averaged  over  all  readers  and  all  lesion  sites,  or  groups  of 
lesion  sites  with  presumed  similar  detection  difficulty. 

The  observed  averages  were  compared  by  the  t-test  for  statistically 
significant  differences. 

RESULTS 

Figure  2  shows  a  representative  example  of  corresponding 
pre-  and  postoperative  radiographs,  and  the  ensuing  subtraction 
image.  The  superimposed  circles  appeared  one  at  a  time  in 
a  random  sequence  over  each  potential  lesion  site.  While  it 
is  nearly  impossible  to  detect  all  lesions  by  comparing  the 
postoperative  (upper  right)  versus  the  preoperative  (upper 
left)  radiograph,  the  lesions  are  easily  detected  in  the 
subtraction  image  (below)  as  dark  blotchy  areas.  In  this 
particular  example  from  a  right  mandible,  lesions  were  induced 
at  sites  1,  3,  4,  and  5.  The  bright  disk-shaped  artifacts 
in  the  radiographs  are  projections  of  spherical  radiopaque 
markers  serving  as  reference  points  to  monitor  the  reproducibility 


of  the  radiographic  projection  geometry. 

The  diagnostic  performance  attained  with  each  modality 
is  shown  in  Table  2.  A  clear  superiority  of  the  subtraction 
technique  over  the  conventional  method  of  comparing  radiographs 
is  evident.  At  the  outset  of  the  investigation,  lesion  sites 
were  grouped  as  shown  in  Table  1  based  upon  the  presumption 
that  members  of  each  group  would  be  subject  to  comparable 
obscuration  due  to  anatomic  overlay.  For  the  conventional 
technique,  the  results  indicate  a  definite  decrease  of  A^ 
for  the  detection  of  interproximal  lesions  as  compared  to  the 
interradicular  and  radicular  groups  (P  <  .01).  No  significant 
differences  existed  between  groups  for  the  subtraction  technique, 
consistent  with  the  premise  that  the  source  of  anatomical 
obscuration  is  cancelled  by  subtraction. 

The  data  also  show  that  pooling  the  11  readers'  raw  data, 
i.e.,  treating  them  as  one  reader  by  merging  their  rating  responses 
leads  to  a  small  depression  of  the  accuracy  index  as  compared 
to  the  average  taken  over  the  individual's  indices.  This  is 

g 

to  be  expected  theoretically,  however,  the  small  difference 
observed  between  the  two  summary  measures  attests  to  the  relative 
uniformity  of  the  decision  criteria  used  among  the  different 
readers . 

Figure  3  shows  the  detection  performance  evaluated  for 
the  total  set  of  lesion  sites.  Every  reader  achieved  a  higher 
accuracy  using  subtraction  images  (P  •  .001).  Also  evident 
is  the  more  uniform  performance  among  the  readers  for  the 
subtraction  as  compared  to  the  conventional  technique. 


1(1 


I  iHiirc  J  displays  the  lt»>i  l.ita  point'-  •ihtainod  by  pool  t  nr, 

the  responses  of  each  reader.  Also  shown  are  the  best -fitted 

lines  for  each  modality  plotted  on  double  probability  coordinates. 

The  ROC  for  the  subtraction  technique  is  seen  to  be  consistently 

above  that  for  the  conventional  technique,  with  respective 

values  of  A  of  0.98  and  0.83. 
z 

A  comparison  of  the  time  intervals  required  to  decide 
whether  at  an  indicated  site  a  lesion  was  present  or  absent 
is  shown  in  Table  3.  In  general,  for  each  of  the  groupings, 
as  well  as  the  total  pool  of  lesion  sites,  the  time  differences 
between  the  two  modalities  were  highly  significant  (P  <  .001). 

The  average  response  times  observed  for  the  conventional  technique 
were  almost  four  times  longer  and  displayed  approximately  twice 
the  standard  errors  as  compared  to  the  subtraction  technique. 
Furthermore,  the  relative  difficulty  of  detecting  lesions 
at  different  sites  was  somewhat  reflected  in  the  times  recorded 
for  the  conventional  technique.  The  average  response  time  for 
lesion  sites  3  and  b,  where  presumably  the  least  amount  of 
obscuration  existed,  was  the  shortest,  and  was  statistically 
different  (P  <  .02)  from  that  obtained  for  sites  1  and  2. 

In  contrast,  the  times  required  in  making  the  decisions  using 
the  subtraction  technique  were  homogeneous  among  the  lesion 
sites  (analysis  of  variance,  P  >  .75). 


DISCUSSION 


Previous  studies  with  skull  phantoms  have  indicated  that 
subtraction  radiography  can  improve  diagnostic  accuracy  when 
compared  with  the  conventional  radiographic  technique.5 
This  investigation  has  confined  these  results  in  a  live 
animal  model.  The  clear  superiority  of  the  subtraction  technique* 
as  demonstrated  in  this  and  other  studies,  is  critically 
dependent  on  the  ability  to  limit  geometric  and  densitometric 
variation  between  radiographs  to  be  compared.  However,  despite 
the  authors'  best  efforts  these  variations  were,  at  times,  quite 
evident  and  dictated  the  two-film  packet  technique.  The  two- 
film  packet  technique  allowed  the  authors  to  continue  making 
radiographs  until  an  empirical  on-site  visual  confirmation 
of  geometric  standardization  could  be  made.  Two  radiographs  per 
site  at  each  observation  interval  was  usually  sufficient. 

The  amount  of  empirically  observed  geometric  variation  over 
the  eight  week  period  during  which  radiographs  were  gathered 
appeared  constant.  Even  a  rigid  registration  method  may, 
over  times  longer  than  those  used  in  this  investigation, 
present  geometric  variation  problems  due  to  normal  minute 
changes  in  tooth  position  which  nay  occur  over  tine  in  some 
animals.  Other  researchers  have  used  a  non-rigid  occlusal 
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registration  with  some  success,  although  in  any  subject 
under  general  anesthesia,  as  well  as  any  animal,  the  use 
of  a  non-rigid  occlusal  registration  would  likely  add  additional 
undesirable  geometric  variation.  The  method  utilized  in  this 
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investigation  to  limit  densitometric  variation  fell  short  of 
the  authors'  goal  of  virtual  elimination.  Indeed,  frequent I) 
the  films  processed  in  a  hand  developer  immediately  following 
exposure  showed  less  densitometric  variation  than  did  the 
duplicate  films  stored  and  processed  under  more  carefully 
controlled  conditions.  Fortunately  the  program  used  for  subtree* 
tion  radiography  can  compensate  for  densitometric  variation.10 
The  choice  of  90  Kvp  exposures  was  made  in  order  to  parallel 
clinical  practice  in  our  area.  60*70  Kvp  would  have  produced 
more  contrast  in  the  radiographic  films  used  for  the  conventional 
technique,  but  likely  would  have  had  little  effect  on  the 
results  produced  after  subtraction  because  the  contrast  under 
the  latter  conditions  can  be  manipulated  electronically. 

The  third  and  fourth  mandibular  premolar  area  was  selected 
for  this  investigation  because,  in  the  dog,  this  region  has 
sufficient  lingual  vestibule  depth  for  parallel  film  placement 
and  there  is  no  interproximal  contact  or  overlap  of  the  third 
premolar  with  the  adjacent  teeth.  Potential  lesion  sites  were 
chosen  to  reflect  incipient  interproximal  periodontal  lesions 
without  cortical  plate  penetration  (Figure  1 ,  Lesions  1  and  2) , 
as  well  as  a  variety  of  overlaying  anatomical  structures  for 
those  lesions  designed  to  penetrate  the  cortical  plate  (Figure 
1,  Lesions  3,  4,  5,  6,  and  7).  The  results  shown  in  Table  2 
indicate  that  the  diagnostic  accuracy  in  detecting  interproximal 
lesions  (sites  1  and  2)  by  the  conventional  technique  was 
substantially  reduced  as  compared  to  the  other  lesion  sites 
(As  •  .77  versus  A^  •  .87).  This  finding  is  in  agreement 


with  other  research  >uur>tii><  ihji  Ioiom*  wt  lataUmt  the 
cortical  plate  are  more  difficult  to  detect  (la  concent  tottal 
radiographs)  than  those  lesions  with  cortical  plate  tavolvowat.1 
Contrasting  with  this,  the  correspond tog  data  from  subtract  too 
radiography  do  not  show  a  specific  association  of  accuracy  with 
lesion  type.  Such  a  result  of  constant  detection  performance 
irrespective  of  aoatonica!  context  should  be  expected  froe  a 
technique  that  is  effective  in  suppressing  structured  noise. 

In  a  clinial  situation,  a  diagnostician  frequently  oust 
nake  a  decision  utilising  less  than  conclusive  evidence.  In 
these  situations  a  clinician  Is  likely  to  show  his  decision 
towards  a  diagnosis,  which  once  nade .  inposes  the  least  ham 
to  the  pet lent  if  the  diagnosis  Is  later  determined  to  be  in* 
correct.  In  an  Investigation  as  this,  tbore  was  no  danger 
to  e  pet lent  In  the  case  of  an  Incorrect  diagnoais,  and  the 
decisions  ware  prestmbly  based  solely  on  the  knowledge  of 
the  prior  probability  of  lesion  occurrence  and  the  information 
derived  from  the  images.  The  diagnostician  was  not  restricted 
by  clinical  pressures  snd  thus  was  free  to  express  his  confidence 
in  each  diagnosis  by  the  rating  scale  provided.  Hence,  this 
technique  permitted  estimating  selective  points  on  the  ROC 
curve  from  the  proportion  of  TP  xnf  FP  decisions  that  would 
be  nade  by  choosing,  in  turn,  each  of  the  possible  rating 
levels  as  decsision  thresholds  between  accepting  or  re)ecting 
the  presence  of  s  lesion. 
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IMtr  clitic*)  eeadmout.  the  decision  il»#rtuU*  or 
operating  point*  m  iW  IOC  caw*  choc—  by  •  diagnostician 
art  Mitlly  ate  katt  btcatt*  they  vary  with  u«  particular 
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Hapwtlc  tail,  tfea  value  Judpaata  node  abate  aorbidity  aa d 
fiaaaeiai  1  apart,  md  aatiaataa  af  disease  prevalence  far  a 
particular  patiaat.  However,  tba  set  lasted  graph*  abaua  ia 
npm  4  still  apply  la  tba  el  laical  eaataxt  bacauaa  tbay 
describe  tba  passible  trade  effa  that  caa  be  aade  batuaaa 
correct  cud  iaaarract  decisions.  txpressod  diffaraatly. 
every  operetta  paiat  that  aay  be  adapted  by  a  diapaasticiaa 
■net  lie  aa  tbs  eppiapriata  ROC  curve  far  a  givaa  radiographic 
tsrhnlquo.  Par  exanple.  if  a  false  peelttvs  (If)  rat*  af  .10 
ia  cliaically  acceptable  ia  a  particular  sitastiea.  subtract  tea 
Tsdiogrepby  uauld  sttaia  a  true  pcs  ltlve  (TP)  prepart  tea  af 
O.H.  aa  eeaparad  ta  tba  ceaveat local  technique  uitb  a  TP 
propectiea  af  0.00.  Or,  if  far  sous  reasea  tba  falaoHMgntlve 
(PH)  decisis*  rata  oust  be  kept  snail,  say  bale*  0.01,  it 
caa  be  see*  fra*  tbe  scales  to  tba  left  and  at  tbe  tap  af  the 
diagran  that  subtraction  radiography  could  provide  a  tree* 
aegative  (TH)  dacisie*  rat*  af  about  0.00,  co*pars<  ta  a 
rot  responding  rata  af  0.00  attainable  with  the  conventional 
technique. 

In  practical  setting,  diagnosticians  any  singly  withhold 
a  definite  respects  in  equivocal  situation  aad  request  further 
diagnostic  evidence.  Usually  it  is  desirable  to  naintaia 
bath  the  probabilities  of  PP  aad  It  responses  below  a  certain 
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Table  3:  Time  Required  to  Perform  Lesion  Detection  Task  (sec.) 
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Post -operative  radiograph.  Below:  The  subtraction  image.  "  ' 
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